The operation of many complex systems requires an operator to monitor status of the system by assimilating a large amount of real-time status data received from various system components. Complex systems of this type include spacecraft operations, industrial plants, financial trading activities, and hospital monitoring of medical data from patients. All of these systems require important decisions to be made, sometimes quickly, in response to a myriad of data. Once the status of the system or the nature of a problem is understood, the system operator takes appropriate actions to implement decisions or to remedy the problem. In extreme cases, these actions may save the system from serious undesirable consequences. It is very important that this complex data be presented clearly and concisely so that the operator can obtain any information needed for diagnostic purposes as easily and quickly as possible.
Existing data display systems often use numerical or two-dimensional displays which require the operator to interpret and analyze the data and its relationship to other data. Some of the features of these kinds of displays are illustrated in FIGS. 1a–1c, which graphically illustrate how the composition of total information conveyed to a user varies over time for various interface types. The total information conveyed in these graphs is equal to the amount of “navigation data” plus the amount of “destination data”. Navigation data is defined as that class of information that orients a user in time, space, location and direction; informs the user of progress in relative and/or absolute terms; and that contains addresses, instruction to proceed, and/or warnings not to proceed. Destination data is defined as that class of information that satisfies the user's desire for information, for example, by answering a question.
Typically, the goal of a user interface is to maximize the volume of available destination data while minimizing the delay in making the destination data available. As shown in FIG. 1a, an operator of a command-driven interface has very little destination data available initially and spends roughly half of the time traversing alphanumeric menus (“navigation data”) to get to the desired data display (“destination data”).
An operator of a menu-driven interface (FIG. 1b) typically can access a larger volume of destination data somewhat faster after first navigating through one or more menu items. However, the operator can become frustrated upon encountering several layers of sub-menus before being able to access the bulk of the destination data. This results in a drop-off (at time D in FIG. 1b) in the amount of available destination data.
In a graphical user interface (“GUI”), the operator traverses through various windows and screens to get at the destination data in a step-wise fashion as illustrated in FIG. 1c. Certain destination data may not be accessible until the operator has stepped through a number of windows, thus diminishing the operator's efficiency. These graphs were presented by Michael Benedikt in the article “Cyberspace: Some Proposals” which appears in the book Cyberspace: First Steps. 
These shortcomings of command, menu and GUI driven interfaces are magnified in data intensive operational environments of the kind described above. The inventors recognized a need in these environments for an improved user interface that makes more information available in less time. Destination data is one example of such data. These problems are well illustrated in the example of spacecraft operation control.
During a typical spacecraft mission, data parameters from many different domains are monitored and/or analyzed in real time to ensure that the spacecraft and its instruments are working properly. A mission control site receives this data from the spacecraft by telemetry. In this setting high volumes of data are received at very frequent intervals. Monitoring this data poses a daunting and complex challenge. This is especially true when multiple spacecraft missions are being monitored simultaneously by a single system.
Early spacecraft data monitoring systems employed a panel of discrete, hardwired lights, each light corresponding to an individual data parameter. A team of human operators (e.g., flight controllers or analysts) monitoring the spacecraft mission, would continually observe the various lights to ensure that the spacecraft and its instruments were operating properly. A trained operator could readily determine whether error or alarm conditions were present simply by scanning the panel for the appropriate color light. For example, a green light might mean that a particular data parameter was within normal ranges, while a red light might indicate that acceptable levels for that parameter had been exceeded. The lights provided a dramatic and immediately comprehensible indication of system status. This early light-panel monitoring system hence had the advantage of providing the operator with an intuitive sense of system health.
As spacecraft and their associated instruments became more sophisticated, light-panel monitoring systems became inadequate to handle the volume and complexity of the telemetry data that was being collected. Light-panel systems also were difficult and labor-intensive to adapt for new types of telemetry data that changed with each mission.
The next generation of data monitoring systems attempted to solve this problem. A computer system (for example, a work station) received telemetry data and displayed it in textual (i.e., alphanumeric) form on the display monitor. FIG. 2 shows a sample output of a typical text-based data monitoring system. The necessarily high volume of telemetry data causes the display screen to become cluttered and difficult to read. An operator relying on the text-based monitoring system in FIG. 2 is required to study and understand large amounts of alphanumeric information to accurately monitor the health of spacecraft. That task requires extensive training and considerable diligence on the operator's part. Even with sufficient training, using a text-based system to monitor data is fatiguing for the operator and frequently results in missed alarm conditions or slow responses to alarm conditions.
In short, text-based data monitoring systems do not generate easily recognizable indicators of changes in system status. Rather, a change in system status (even a potentially catastrophic one) is signalled merely by changing one or more alphanumeric characters to other lphanumeric characters on a display screen. The screen typically contains several hundred such alphanumeric characters, and even the most vigilant human operator can occasionally overlook an important alarm condition. Thus, there is a need for a system that can present a large amount of information to a user in a way that is readily understandable and in a way that conveys important conditions in a very noticeable way.
The data monitoring system of the present invention uses a three dimensional simulated space representation, (sometimes called a “cyberspace representation,”), to interface with, and to communicate complex, real-time information to an operator. A cyberspace representation utilizes various elements (e.g., time, space, sound, travel, and user presence in the computer environment) to convey information to the operator. The root word “cyber” comes from the Greek word “kybernan” which means “to steer or control.” Literally, cyberspace means to steer or control space.
The cyberspace data monitoring system of the present invention may be implemented in conjunction with artificial reality or virtual reality systems. The cyberspace interface of the invention provides a graphically-oriented user interface where the operator is figuratively positioned within the user interface environment. The invention uses a visual representation to communicate information to the user. The interface appears to the user as a perspective view of a three-dimensional space from a particular vantage point. The user may move around within the three dimensional space, and thus change his view through manipulation of a cursor input device. As the user navigates through the three-dimensional space, the display changes to reflect the operator's new position within the space. This movement may reveal objects that were not previously visible from the operator's prior vantage point.
The use of this cyberspace representation allows the present data monitoring system to realize a “textless” or “minimal text” display that allows a human operator to assimilate and comprehend large quantities of data at a rate and to a degree much greater than previously possible. This is because the user interface of the invention is in a form similar to that used to comprehend real-world events. Rather than representing information in the form of multiple rows and columns of alphanumeric characters, data is depicted in the form of graphic symbols or objects which are positioned within a three-dimensional virtual universe. The respective dimensions of the virtual universe correspond to categories of information from various domains. For example, in one embodiment of the invention, these categories may include: the various spacecraft missions being monitored, various user-defined categories of data parameters that are relevant to the application, attitude control system, propulsion, Alarm Log and User-Defined. The Alarm Log category records all anomalous data values until they are acknowledged by the user. The User-Defined category allows copies of data values from any other category to be grouped together. A third dimension may correspond to certain user-defined alarm limits of state value for each of the data parameter categories.
The shape, motion, position, color and behavior of the graphic symbols are configured to convey an extensive amount of information about the current values of the data parameters to the human operator in an easily understandable and instantly recognizable manner.
FIG. 1d shows the Total Information to User graph for a cyberspace-driven interface. A cyberspace-driven interface does not require an operator to enter alphanumeric commands or to traverse menu items or windows before destination data is available. Rather, as illustrated in FIG. 1d by the three exemplary curves of cyberspace-driven interfaces, the operator has essentially instantaneous access to a considerable amount of destination data. This results in a greater amount of usable information being conveyed to the operator in less time.
Another advantage provided by the cyberspace-driven system of the invention is that its visual abstraction of data allows an operator to assess the qualitative status of the various spacecraft with not much more than a mere glance at the display screen. For example, in one embodiment of the cyberspace system of the invention, information about alarm severity is conveyed by the height of a pole extending from the graphical objects. In a text-based system, in contrast, severity information may be obtained only by manually referring to the alphanumeric alarm limits of the parameters of interest.
The cyberspace system also permits more data to be displayed on the screen at any one time. Only a part of one screen is used in the cyberspace system (in contrast to the multiple screens that are used for text-based systems), hence all of the telemetry data for an entire system can be displayed unambiguously in a fraction of the screen real-estate that has been previously required.
Yet another advantage of the cyberspace data monitoring system is that it allows multiple systems to be monitored simultaneously, and further allows related data parameters to be grouped (and re-grouped) by the operator logically in space. Thus, the cyberspace system is easily tailored to the specific needs of a particular system or to the preferences of a particular operator.
Standard computer graphics manipulation techniques are used to facilitate navigation through this virtual universe of data to alter the relative vantage point within the universe. Hence the operator may move between different monitored systems (such as different spacecraft missions) to inspect a particular data parameter or group of parameters in more detail than might be possible at a different vantage point within the universe.
The cyberspace data representation provided by the invention eliminates the need for an operator to read large volumes of textual material in order to determine system status. Rather, merely by scanning the attributes of the graphic symbols displayed on the screen, an operator gleans an intuitive understanding of the system status that extends far beyond the knowledge that could be imparted by a visual inspection of either a light-panel display or a text-based display.
Other advantages of the cyberspace data monitoring system include the following: an operator is able to recognize impending alarm conditions earlier and with greater reliability due to the intuitive nature of the cyberspace interface; operators may be trained faster and with greater efficiency; a plurality of sub-systems can be operated as a single system instead of as collections of sub-systems because the status of the entire system can be understood from one display; fewer operators may be required by enabling a single operator to effectively monitor larger components of a system; operators can become aware of problem situations before actual alarms occur and without needing to manually assess collected trend data; and operations differences between non-identical systems can be minimized.